Adaptive Regularization in Neural Network Modeling

. In this paper we address the important problem of optimizing regularization parameters in neural network modeling. The suggested optimization scheme is an extended version of the recently presented algorithm [24]. The idea is to minimize an empirical estimate -- like the cross-validation estimate -- of the generalization error with respect to regularization parameters. This is done by employing a simple iterative gradient descent scheme using virtually no additional programming overhead compared to standard training. Experiments with feed-forward neural network models for time series prediction and classification tasks showed the viability and robustness of the algorithm. Moreover, we provided some simple theoretical examples in order to illustrate the potential and limitations of the proposed regularization framework. 1 Introduction Neural networks are flexible tools for time series processing and pattern recognition. By increasing the number of hidden neurons in a 2-layer architec..

An inventory control project in a major Danish company using compound renewal demand models

We describe the development of a framework to compute the optimal inventory policy for a large spare-parts’ distribution centre operation in the RA division of the Danfoss Group in Denmark. The RA division distributes spare parts worldwide for cooling and A/C systems. The warehouse logistics operation is highly automated. However, the procedures for estimating demands and the policies for the inventory control system that were in use at the beginning of the project did not fully match the sophisticated technological standard of the physical system. During the initial phase of the project development we focused on the fitting of suitable demand distributions for spare parts and on the estimation of demand parameters. Demand distributions were chosen from a class of compound renewal distributions. In the next phase, we designed models and algorithmic procedures for determining suitable inventory control variables based on the fitted demand distributions and a service level requirement stated in terms of an order fill rate. Finally, we validated the results of our models against the procedures that had been in use in the company. It was concluded that the new procedures were considerably more consistent with the actual demand processes and with the stated objectives for the distribution centre. We also initiated the implementation and integration of the new procedures into the company’s inventory management systemBase-stock policy; compound distribution; fill rate; inventory control; logistics; stochastic processes

Weighted Modal Transition Systems

Specification theories as a tool in model-driven development processes of component-based software systems have recently attracted a considerable attention. Current specification theories are however qualitative in nature, and therefore fragile in the sense that the inevitable approximation of systems by models, combined with the fundamental unpredictability of hardware platforms, makes it difficult to transfer conclusions about the behavior, based on models, to the actual system. Hence this approach is arguably unsuited for modern software systems. We propose here the first specification theory which allows to capture quantitative aspects during the refinement and implementation process, thus leveraging the problems of the qualitative setting. Our proposed quantitative specification framework uses weighted modal transition systems as a formal model of specifications. These are labeled transition systems with the additional feature that they can model optional behavior which may or may not be implemented by the system. Satisfaction and refinement is lifted from the well-known qualitative to our quantitative setting, by introducing a notion of distances between weighted modal transition systems. We show that quantitative versions of parallel composition as well as quotient (the dual to parallel composition) inherit the properties from the Boolean setting.Comment: Submitted to Formal Methods in System Desig

On design and evaluation of tapped-delay neural network architectures

We address pruning and evaluation of Tapped-Delay Neural Networks for the sunspot benchmark series. It is shown that the generalization ability of the networks can be improved by pruning using the Optimal Brain Damage method of Le Cun, Denker and Solla. A stop criterion for the pruning algorithm is formulated using a modified version of Akaike's Final Prediction Error estimate. With the proposed stop criterion the pruning scheme is shown to produce succesful architectures with a high yield. I. Introduction Needless to say, processing of time series is an important application area for neural networks, and the quest for application-specific architectures penetrates current network research. While the ultimate tool may be fully recurrent architectures, many problems arise during adaptation of these. Even worse, the generalization properties of recurrent networks are not well understood, hence, model optimization is difficult. However, the conventional Tapped-Delay Neural Net (TDNN) [11..